Objective¶
The objective of this project is to classify a given silhouette as one of three types of vehicle, using a set of features extracted from the silhouette. The vehicle may be viewed from one of many different angles.
Plans to Execute¶
- Load dataset
- Check the dataset information and distribution
- Check missing values
- visualize the feature distributions and relationship between the features
- Outliers analysis
- Training and testing on raw dataset
- Principal Component Analysis
- Training and testing on transformed features(Principal Components)
- Feature selection using Extra Tree Classfier and Random Forest (Not Implemented in this project)
- K-Means clustering on full data set
- Ensemble classification and cluster results and retrain the results using Logistic Regression(Not Implemented in this project)
- Summary
Exploratory Data Analysis¶
In [474]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
Load Dataset¶
In [476]:
dataset = pd.read_csv("vehicle.csv")
dataset.head()
Out[476]:
In [477]:
dataset.info()
All the independent features are of numeric type¶
In [479]:
dataset.isnull().sum()
Out[479]:
There are a few missing values in almost ~15 columns. We need to find the row having one or more missing value columns and try to fill to it by using mean or median. The manipulated records should be kept in training data.¶
In [478]:
dataset.describe().T
Out[478]:
In [482]:
sns.pairplot(dataset)
Out[482]:
In [486]:
plt.subplots(figsize=(10,10))
sns.heatmap(dataset.corr(), annot = True)
plt.show()
Most of the independent variables has linear relationship among themselves. This will impact the performance of linear models like SVM, Logistic Regression etc. So we need to do PCA to get rid of linear relation among the independent features. Based on how much variance is captured by the principal components, we can reduce the dimension feature space.¶
Detailed Analysis of features¶
Circularity and Compactness¶
In [488]:
sns.scatterplot(dataset["compactness"], dataset["circularity"],hue=dataset["class"])
Out[488]:
From the above figure, I doubt the following things
- Outlier in circularity and copactness for class "bus" because the green dots are widely scattered
- Linear relationship between circularity and compactness doesn't exist for class "van"
- If we manage outliers for class "bus", the linear relationship will be reduced for class "bus" as well
In [492]:
sns.boxplot(y = dataset["circularity"], x=dataset["class"])
Out[492]:
In [490]:
sns.boxplot(y = dataset["compactness"], x=dataset["class"])
Out[490]:
Outliers exist in circularity values for class "bus"
It's difficult to differentiate van and bus by checking the "circularity", but car has high circularity value. Since there are outliers in Circulariy for "bus" class, there are chances to misclassify "bus" as "car". So we need to handle outliers prior to training.
Circularity and Distance Cirucularity¶
In [493]:
sns.scatterplot(dataset["compactness"], dataset["distance_circularity"],hue=dataset["class"])
Out[493]:
Same observation as that of Compactness and Circularity
In [509]:
f,axes =plt.subplots(nrows=3,ncols=3, figsize=(15,15))
sns.scatterplot(dataset["compactness"], dataset["scatter_ratio"],hue=dataset["class"],ax=axes[0, 0])
sns.scatterplot(dataset["compactness"], dataset["elongatedness"],hue=dataset["class"],ax=axes[0, 1])
sns.scatterplot(dataset["compactness"], dataset["pr.axis_rectangularity"],hue=dataset["class"],ax=axes[0, 2])
sns.scatterplot(dataset["compactness"], dataset["max.length_rectangularity"],hue=dataset["class"],ax=axes[1, 0])
sns.scatterplot(dataset["compactness"], dataset["scaled_variance"],hue=dataset["class"],ax=axes[1, 1])
sns.scatterplot(dataset["compactness"], dataset["scaled_radius_of_gyration"],hue=dataset["class"],ax=axes[1, 2])
sns.scatterplot(dataset["compactness"], dataset["scaled_variance.1"],hue=dataset["class"],ax=axes[2, 0])
sns.scatterplot(dataset["circularity"], dataset["distance_circularity"],hue=dataset["class"],ax=axes[2,1])
sns.scatterplot(dataset["circularity"], dataset["scatter_ratio"],hue=dataset["class"],ax=axes[2,2])
Out[509]:
Higher values of scatter_ratio for cars
In [517]:
f,axes =plt.subplots(nrows=3,ncols=3, figsize=(15,15))
sns.scatterplot(dataset["circularity"], dataset["scatter_ratio"],hue=dataset["class"],ax=axes[0, 0])
sns.scatterplot(dataset["circularity"], dataset["elongatedness"],hue=dataset["class"],ax=axes[0, 1])
sns.scatterplot(dataset["circularity"], dataset["pr.axis_rectangularity"],hue=dataset["class"],ax=axes[0, 2])
sns.scatterplot(dataset["circularity"], dataset["max.length_rectangularity"],hue=dataset["class"],ax=axes[1, 0])
sns.scatterplot(dataset["circularity"], dataset["scaled_variance"],hue=dataset["class"],ax=axes[1, 1])
sns.scatterplot(dataset["circularity"], dataset["scaled_radius_of_gyration"],hue=dataset["class"],ax=axes[1, 2])
sns.scatterplot(dataset["circularity"], dataset["scaled_variance.1"],hue=dataset["class"],ax=axes[2, 0])
sns.scatterplot(dataset["radius_ratio"], dataset["max.length_rectangularity"],hue=dataset["class"],ax=axes[2,1])
sns.scatterplot(dataset["elongatedness"], dataset["scatter_ratio"],hue=dataset["class"],ax=axes[2,2])
Out[517]:
Analysis of Outliers¶
In [515]:
f,axes =plt.subplots(nrows=3,ncols=3, figsize=(15,15))
sns.boxplot(y=dataset["compactness"], x=dataset["class"],ax=axes[0, 0])
sns.boxplot(y=dataset["elongatedness"],x=dataset["class"],ax=axes[0, 1])
sns.boxplot(y=dataset["pr.axis_rectangularity"],x=dataset["class"],ax=axes[0, 2])
sns.boxplot(y=dataset["max.length_rectangularity"],x=dataset["class"],ax=axes[1, 0])
sns.boxplot(y=dataset["scaled_variance"],x=dataset["class"],ax=axes[1, 1])
sns.boxplot(y=dataset["scaled_radius_of_gyration"],x=dataset["class"],ax=axes[1, 2])
sns.boxplot(y=dataset["scaled_variance.1"],x=dataset["class"],ax=axes[2, 0])
sns.boxplot(y=dataset["distance_circularity"],x=dataset["class"],ax=axes[2,1])
sns.boxplot(y=dataset["scatter_ratio"],x=dataset["class"],ax=axes[2,2])
Out[515]:
Most of the features for "bus" contain extreme values. So scaling of the dataset will help to handle the outlier to some extend
In [516]:
f,axes =plt.subplots(nrows=3,ncols=3, figsize=(15,15))
sns.boxplot(y=dataset["circularity"], x=dataset["class"],ax=axes[0, 0])
sns.boxplot(y=dataset["distance_circularity"],x=dataset["class"],ax=axes[0, 1])
sns.boxplot(y=dataset["pr.axis_aspect_ratio"],x=dataset["class"],ax=axes[0, 2])
sns.boxplot(y=dataset["max.length_aspect_ratio"],x=dataset["class"],ax=axes[1, 0])
sns.boxplot(y=dataset["scaled_radius_of_gyration.1"],x=dataset["class"],ax=axes[1, 1])
sns.boxplot(y=dataset["skewness_about.1"],x=dataset["class"],ax=axes[1, 2])
sns.boxplot(y=dataset["skewness_about.2"],x=dataset["class"],ax=axes[2, 0])
sns.boxplot(y=dataset["hollows_ratio"],x=dataset["class"],ax=axes[2,1])
Out[516]:
Hollows ratio for "bus" is having lower values compared to"van" and "car"
max_length_aspect ratio is an important feature for identifying bus from other two classes.
distance_circularity is higher for car.
lower value of scaled_varience points to the class -"van"
Elongateness is higher for "van"
Training and Performance Evaluation¶
In [431]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
# module for splitting the entire dataset into training and testing dataset
from sklearn.model_selection import train_test_split
#import LogisticReGression Model
from sklearn.linear_model import LogisticRegression
#import NaiveBayes model
from sklearn.naive_bayes import GaussianNB
#import KNN model
from sklearn.neighbors import KNeighborsClassifier
#import SVM model
from sklearn.svm import SVC
#Import metrics modulel to calculate model performance
from sklearn import metrics
#import resample module for upsampling
from sklearn.utils import resample
# module for evaluating the feature dependecies with target variable
import statsmodels.formula.api as sm
# module for cross validation
from sklearn.model_selection import cross_val_score
# module for AUC-ROC Calculation
from sklearn.metrics import roc_curve, auc
# module for SVM hyperparameter tuning
from sklearn.model_selection import GridSearchCV,RandomizedSearchCV
# module for ensembled models
from sklearn.ensemble import RandomForestClassifier, BaggingClassifier, AdaBoostClassifier, VotingClassifier,GradientBoostingClassifier,ExtraTreesClassifier
# module for decision tree
from sklearn.tree import DecisionTreeClassifier
#
from xgboost import XGBClassifier
from catboost import CatBoostClassifier
import lightgbm as lgb
from sklearn.preprocessing import StandardScaler
from sklearn.cluster import KMeans
import random
import warnings
warnings.filterwarnings('ignore')
%matplotlib inline
class tuning_params:
def __init__(self):
self.random_forest = None
self.adaboost = None
self.svm = None
class feature_selection:
def __init__(self, features):
self.feature_importance_metrics = pd.DataFrame()
self.feature_importance_metrics["features"] = features
self.feature_importance_metrics["NetImportance"] = np.zeros(features.length)
def add_to_metrics(feature_importance,model_name):
self.feature_importance_metrics = pd.merge(self.feature_importance_metrics,feature_importance,on=["features"])
self.feature_importance_metrics["NetImportance"] = self.feature_importance_metrics.apply(lambda x: (x["NetImportance"] + x[model_name])/2)
def get_features(threshold):
return self.feature_importance_metrics[(self.feature_importance_metrics["NetImportance"]>threshold)]["features"]
hyperparams = tuning_params()
def drop_columns(df, col_names,inplace=False):
print("--------Dropping columns-----")
return df.drop(col_names,inplace = inplace, axis=1)
def transform_data(df):
# Create a new column CCUsage from CCAvg column
df["CCUsage"] = df.apply(lambda x: 1 if x.CCAvg < 3 else 2 if (x.CCAvg >= 3 and x.CCAvg <6) else 3 if (x.CCAvg >= 6 and x.CCAvg <8) else 4, axis=1)
# Create a new column IncomeStatus from Income
df["IncomeStatus"] =df.apply(lambda x: 1 if x.Income <110 else 2 if x.Income>=110 and x.Income <180 else 3,axis=1)
# Create a new column HasMortage from Mortage
df["HasMortgage"] = df.apply(lambda x: 0 if x.Mortgage == 0 else 1,axis=1)
# Scale the Age Column as a new column
df["Age_Scaled"] = df.apply(lambda x: (x.Age - df["Age"].mean())/df["Age"].std(),axis=1)
# New column after scaling experience column
df["Experience_Scaled"] = df.apply(lambda x: (x.Age - df["Experience"].mean())/df["Experience"].std(),axis=1)
return df
def get_features_and_target(df,target_column_name):
X=df.drop([target_column_name],axis=1)
y=df[target_column_name]
return X,y
def get_onehot_encoded_features(X):
categoryList = list(X.select_dtypes(include=['object']).columns)
dummies = pd.get_dummies(X[categoryList], prefix= categoryList)
X.drop(categoryList, axis=1, inplace = True)
X=pd.concat([X,dummies], axis =1)
return X
def handle_missing_values(df, mode):
MISSING_VALUE_HANDLER ={
"median": df.fillna(df.median()),
"mode": df.fillna(df.median()),
"mean": df.fillna(df.mean())
}
return MISSING_VALUE_HANDLER.get(mode)
def upsample_data(df,target_column_name, minority_class, majority_class, number_of_samples):
majority=df[df[target_column_name] == majority_class]
minority=df[df[target_column_name] == minority_class]
minority_upsampled = resample(minority,replace=True,n_samples = number_of_samples)
upsampled=pd.concat([minority_upsampled,majority])
return upsampled
def train_model(algorithm_name, model, X_train, X_test, y_train, y_test):
model.fit(X_train, y_train)
print("#############Evaluating the Performance of ",algorithm_name,"#############")
evaluate_performance(model, X_train,y_train, X_test, y_test)
print("#############Evaluation Completed for ",algorithm_name," ###################")
def evaluate_performance(model, X_train, y_train, X_test, y_test):
y_predict = model.predict(X_test)
print("\n")
print("Confusion Metrics:")
print(metrics.confusion_matrix(y_test, y_predict))
print("\n")
print("Model Performance Summary:")
print(metrics.classification_report(y_test, y_predict))
print("Evaluating the Cross Validation score")
#print(X_train.shape)
#print(y_train.shape)
accuracies= cross_val_score(estimator = model, X = X_train, y = y_train, cv = 5)
accuracies_mean=accuracies.mean()*100
print("Mean Accuracy:",accuracies_mean)
accuracies_std=accuracies.std()*100
print("Standard Deviation:",accuracies_std)
#plot_auc_roc_curve(y_test, y_predict)
def plot_auc_roc_curve(y_test,y_pred):
false_positive_rate, true_positive_rate, thresholds = roc_curve(y_test, y_pred)
roc_auc = auc(false_positive_rate, true_positive_rate)
plt.title('Receiver Operating Characteristic')
plt.plot(false_positive_rate, true_positive_rate, 'b',
label='AUC = %0.2f'% roc_auc)
plt.legend(loc='lower right')
plt.plot([0,1],[0,1],'r--')
plt.xlim([-0.1,1.2])
plt.ylim([-0.1,1.2])
plt.ylabel('True Positive Rate')
plt.xlabel('False Positive Rate')
plt.show()
def evaluate_bestK(X_train,y_train, X_test, y_test):
error_rate = []
for i in range(1,50):
knn = KNeighborsClassifier(n_neighbors = i)
knn.fit(X_train,y_train)
pred_i = knn.predict(X_test)
error_rate.append(np.mean(pred_i != y_test))
plt.figure(figsize=(10,6))
plt.plot(range(1,50), error_rate,color = "green", linestyle="dashed", marker = "o", markerfacecolor='red',markersize=10)
plt.title("Error Rate vs K")
plt.xlabel('K')
plt.ylabel('Error Rate')
plt.show()
return error_rate.index(min(error_rate)) + 1
def random_forest_hyperparams():
# Number of trees in random forest
n_estimators = [int(x) for x in np.linspace(start = 50, stop = 200, num = 5)]
# Number of features to consider at every split
max_features = ['auto', 'sqrt']
# Maximum number of levels in tree
max_depth = [int(x) for x in np.linspace(10, 20, num = 5)] ## change 10,20 and 2
max_depth.append(None)
# Minimum number of samples required to split a node
min_samples_split = [2, 5, 10,15]
# Minimum number of samples required at each leaf node
min_samples_leaf = [1, 2, 4,10]
# Method of selecting samples for training each tree
bootstrap = [True, False]
return n_estimators, max_features,max_depth, min_samples_split, min_samples_leaf, bootstrap
def adaboost_hyperparams():
# Number of trees in random forest
n_estimators = [int(x) for x in np.linspace(start = 50, stop = 200, num = 5)] ## play with start and stop
learning_rate = [1,0.1,0.01,0.001,0.5,0.05,0.005]
return n_estimators, learning_rate
def get_model_turning_params(modelname):
TUNING_PARAM_VALUES = {
"random_forest":
random_forest_hyperparams(),
"ada_boost":
adaboost_hyperparams()
}
return TUNING_PARAM_VALUES.get(modelname)
def initalize_tuning_params():
tuning_params_svm = {'C': [1,0.01,0.05,1], 'gamma': [1,0.1,0.01,0.002,0.0005,0.001,0.0001], 'kernel': ['rbf','linear']}
n_estimators, max_features,max_depth, min_samples_split, min_samples_leaf, bootstrap = get_model_turning_params("random_forest")
tuning_params_randomforest = {'n_estimators': n_estimators,
'max_features': max_features,
'max_depth': max_depth,
'min_samples_split': min_samples_split,
'min_samples_leaf': min_samples_leaf,
'bootstrap': bootstrap}
n_estimators, learning_rate = get_model_turning_params("ada_boost")
tuning_params_adaboost = {'n_estimators': n_estimators,
'max_features': max_features,
'learning_rate': learning_rate}
hyperparams.random_forest = tuning_params_randomforest
hyperparams.adaboost = tuning_params_adaboost
hyperparams.svm = tuning_params_svm
def evaluate_logisticregression_performance(X_train,y_train, X_test, y_test):
model = LogisticRegression()
train_model("Logistic Regression", model, X_train, X_test, y_train, y_test)
def evaluate_naivebayes_performance(X_train,y_train, X_test, y_test):
model = GaussianNB()
train_model("Naive Bayes", model, X_train, X_test, y_train, y_test)
def evaluate_knn_performance(X_train,y_train, X_test, y_test):
optimal_neighbor = evaluate_bestK(X_train, y_train, X_test, y_test)
print("Optimal value of K is: ",optimal_neighbor)
model = KNeighborsClassifier(n_neighbors= optimal_neighbor, weights="distance")
train_model("KNN", model, X_train, X_test, y_train, y_test)
def do_hyperparams_tuning_svm(X_train,y_train, X_test, y_test):
param_grid = {'C': [1,0.01,0.05,1], 'gamma': [1,0.1,0.01,0.002], 'kernel': ['rbf','linear']}
model = GridSearchCV(SVC(),param_grid,refit=True,verbose=3)
model.fit(X_train,y_train)
return model.best_estimator_
def evaluate_svm_performance(X_train,y_train, X_test, y_test):
svc_model = SVC()
train_model("SVM", svc_model, X_train, X_test, y_train, y_test)
def evaluate_adaboost_performance(X_train,y_train, X_test, y_test):
model = AdaBoostClassifier(DecisionTreeClassifier(),n_estimators = 3, learning_rate = 0.001)
train_model("AdaBoost", model, X_train, X_test, y_train, y_test)
def evaluate_gradientboost_performance(X_train,y_train, X_test, y_test):
model = GradientBoostingClassifier(learning_rate=0.01,random_state=1)
train_model("GradientBoost", model, X_train, X_test, y_train, y_test)
def evaluate_metabagging_performance(X_train,y_train, X_test, y_test):
model = BaggingClassifier(DecisionTreeClassifier(random_state=1))
train_model("MetaBagging", model, X_train, X_test, y_train, y_test)
def evaluate_randomforest_performance(X_train,y_train, X_test, y_test):
classifier = RandomForestClassifier(n_estimators = 10, criterion='entropy')
train_model("RandomForest", classifier, X_train, X_test, y_train, y_test)
def do_hyperparms_tuning_rf(X_train,y_train, X_test, y_test):
model = RandomizedSearchCV(estimator = RandomForestClassifier(), param_distributions = hyperparams.random_forest, n_iter = 100, cv = 3,
verbose=2, random_state=42, n_jobs = -1)
model.fit(X_train,y_train)
return model.best_estimator_
def evaluate_xgboost_performance(X_train,y_train, X_test, y_test):
model = XGBClassifier()
train_model("XGBoost", model, X_train, X_test, y_train, y_test)
def evaluate_catboost_performance(X_train,y_train, X_test, y_test):
model = CatBoostClassifier(iterations=100, depth=3, learning_rate=0.1, loss_function='Logloss')
train_model("Catboost", model, X_train, X_test, y_train, y_test)
def evaluate_extratreeclassifier_performance(X_train,y_train, X_test, y_test):
model = ExtraTreesClassifier(n_estimators=700,criterion= 'entropy',min_samples_split= 5,
max_depth= 50, min_samples_leaf= 5)
train_model("Extra Tree Classifier", model, X_train, X_test, y_train, y_test)
def show_feature_importance(X,model,threshold):
df= pd.DataFrame()
df['feature'] = X.columns
df['Importance Index']= model.feature_importances_
print(df)
#def evaluate_kmeans_clustering(data):
#kmeans = KMeans(n_clusters = 3, init = 'k-means++', random_state = 42, max_iter=300,n_init=3) ## n_init number of centroids initialized
#y_pred = kmeans.fit_predict(data)
#data['Cluster'] = data['class'].apply(lambda r: 0 if r["class"] == "van" else 1 if r["class"] == "car" else 2,axis=1)
#print(confusion_matrix(df['Cluster'],y_kmeans))
#print(classification_report(df['Cluster'],y_kmeans))
def do_pca(df_scaled):
cov_matrix = np.cov(df_scaled.T)
eigenvalues, eigenvectors = np.linalg.eig(cov_matrix)
print("EigenValues Size: ",eigenvalues.size)
print("EigenVectors: \n",eigenvalues)
eig_pairs = [(eigenvalues[index], eigenvectors[:,index]) for index in range(len(eigenvalues))]
eig_pairs.sort()
eig_pairs.reverse()
eigvalues_sorted = [eig_pairs[index][0] for index in range(len(eigenvalues))]
eigvectors_sorted = [eig_pairs[index][1] for index in range(len(eigenvalues))]
tot = sum(eigenvalues)
var_explained = [(i / tot) for i in sorted(eigenvalues, reverse=True)]
cum_var_exp = np.cumsum(var_explained)
print("Cumulative Variance: \n",cum_var_exp)
plt.bar(range(1,19), var_explained, alpha=0.5, align='center', label='individual explained variance')
plt.step(range(1,19),cum_var_exp, where= 'mid', label='cumulative explained variance')
plt.ylabel('Explained variance ratio')
plt.xlabel('Principal components')
plt.legend(loc = 'best')
plt.show()
Load Dataset¶
In [432]:
vehicles_df = pd.read_csv("vehicle.csv")
In [433]:
vehicles_df.head()
Out[433]:
No Categorical values in the data other than the class column
Get features and target in separate data frames¶
In [434]:
X, y = X,y = get_features_and_target(vehicles_df,"class")
In [435]:
X.shape
Out[435]:
In [436]:
y.shape
Out[436]:
Handling Missing values¶
In [437]:
X.isnull().sum()
Out[437]:
There are missing values in different columns. I will fill the missing values with MEDIAN. The manipulated records should be in training data, so these records should be separated from X before spliting the data into testinga nd training.¶
In [438]:
vehicles_df_clean = vehicles_df.fillna(X.median())
vehicles_df_filled = vehicles_df_clean[(vehicles_df.isnull().any(axis=1))]
vehicles_df_original = vehicles_df.dropna()
In [439]:
vehicles_df_original.shape
Out[439]:
In [440]:
vehicles_df_filled.shape
Out[440]:
~33 records were filled with median value. That's around 4% of the actual data. Since I don't want to include these records in test data, I am splitting the X_Original(records without any of missing values) into traing and testing. In order to compensate train vs test data count in 70:30, I will use test_rate as 34%. Later I will merge the manipulated records to the traing set¶
Get Features and target¶
In [441]:
X_original,y_original = get_features_and_target(vehicles_df_original,"class")
X_filled,y_filled = get_features_and_target(vehicles_df_filled,"class")
Split dataset into training and testing data¶
In [442]:
X_train, X_test, y_train, y_test = train_test_split(X_original,y_original,test_size=0.34,random_state=0)
Merge manipulated records with training set¶
In [443]:
X_train=pd.concat([X_train,X_filled])
y_train =pd.concat([y_train,y_filled])
In [444]:
print(X_train.shape)
print(X_test.shape)
In [445]:
print(y_train.shape)
print(y_test.shape)
Checking the target class distribution¶
In [446]:
y_train.value_counts()
Out[446]:
In [447]:
y_test.value_counts()
Out[447]:
Target class distribution is not bad¶
Initialize tuning params¶
In [448]:
initalize_tuning_params()
Logistic Regression¶
In [449]:
evaluate_logisticregression_performance(X_train,y_train,X_test,y_test)
The F1 scores for classes are pretty good, but the model is Overfit(more than 3% of Standard deviation observed)¶
Naive Bayes¶
In [450]:
evaluate_naivebayes_performance(X_train,y_train,X_test,y_test)
The results are not good and the model is overfit¶
KNN¶
In [451]:
evaluate_knn_performance(X_train,y_train,X_test,y_test)
Optimal number of neigbours for prediction is 1.¶
SVM¶
In [452]:
evaluate_svm_performance(X_train,y_train,X_test,y_test)
SVM model did not perform well with default values of hyper parameters¶
SVM(with hyperparameter tuning)¶
In [453]:
model=do_hyperparams_tuning_svm(X_train,y_train,X_test,y_test)
In [454]:
train_model("SVM (with hyperparam tuning)", model, X_train, X_test, y_train, y_test)
The model performed well with more than 97% of F1 score. But the model seems to be overfit because of high STD. This could be due to multicollenearity between the independent features¶
AdaBoost¶
In [455]:
evaluate_adaboost_performance(X_train,y_train,X_test,y_test)
Gradient Boost¶
In [456]:
evaluate_gradientboost_performance(X_train,y_train,X_test,y_test)
Meta Bagging¶
In [457]:
evaluate_metabagging_performance(X_train,y_train,X_test,y_test)
Random Forest¶
In [458]:
evaluate_randomforest_performance(X_train,y_train,X_test,y_test)
Random Forest with Hyperparameter tuning¶
In [459]:
model = do_hyperparms_tuning_rf(X_train,y_train,X_test,y_test)
train_model("RandomForest (With hyperparam tuning)", model, X_train, X_test, y_train, y_test)
Random Forest performed very well with default data. Class level F1 score is ~95%. More over the standard deviation is very low.¶
PCA - Handle Multicollinearity and reduce dimensions¶
In [460]:
sc= StandardScaler()
df_scaled = sc.fit_transform(X_train)
do_pca(df_scaled)
98% of the variance is captured by first 8 principle components. So I am taking the number of features as 8¶
In [461]:
P_reduce = np.array(eigvectors_sorted[0:8])
X_train_reduced = np.dot(df_scaled,P_reduce.T)
X_train_reduced = pd.DataFrame(X_train_reduced)
X_train_reduced.head()
Out[461]:
In [463]:
X_train_reduced.shape
Out[463]:
In [464]:
sns.pairplot(X_train_reduced)
Out[464]:
From the above pairplot, it's clear that the multicollinearity between the features are completely vanished after transforming the features into principal components.¶
In [465]:
sc= StandardScaler()
df_scaled_test = sc.fit_transform(X_test)
X_test_reduced = np.dot(df_scaled_test,P_reduce.T)
X_test_reduced = pd.DataFrame(X_test_reduced)
X_test_reduced.head()
Out[465]:
In [466]:
X_test_reduced.shape
Out[466]:
Re-train the classification models using transformed features:¶
Logistic Regression¶
In [467]:
evaluate_logisticregression_performance(X_train_reduced,y_train,X_test_reduced,y_test)
Naive Bayes¶
In [468]:
evaluate_naivebayes_performance(X_train_reduced,y_train,X_test_reduced,y_test)
KNN¶
In [469]:
evaluate_knn_performance(X_train_reduced,y_train,X_test_reduced,y_test)
SVM¶
In [470]:
evaluate_svm_performance(X_train_reduced,y_train,X_test_reduced,y_test)
SVM (with Hyperparameter tuning)¶
In [471]:
model=do_hyperparams_tuning_svm(X_train_reduced,y_train,X_test_reduced,y_test)
In [472]:
train_model("SVM (with hyperparam tuning)", model, X_train_reduced, X_test_reduced, y_train, y_test)
Gradient Boost¶
In [473]:
evaluate_gradientboost_performance(X_train_reduced,y_train,X_test_reduced,y_test)
In [ ]:
Feature Selection¶
NOT IMPLEMENT EDEXCEPTION() :)¶
Clustering¶
In [521]:
vehicles_df["class"] = vehicles_df.apply(lambda r: 0 if r["class"] == "van" else 1 if r["class"] == "car" else 2,axis=1)
In [522]:
vehicles_df = vehicles_df.fillna(X.median())
In [523]:
vehicles_df.head()
Out[523]:
In [270]:
def evaluate_kmeans_clustering(data):
kmeans = KMeans(n_clusters = 3, init = 'k-means++', random_state = 42, max_iter=300)
y_pred = kmeans.fit_predict(data.drop("class",axis=1))
print(y_pred)
#data['Cluster'] = data.apply((lambda r: 0 if r["class"] == "van" else 1 if r["class"] == "car" else 2),axis=1)
print(metrics.confusion_matrix(data['class'],y_pred))
print(metrics.classification_report(data['class'],y_pred))
In [271]:
evaluate_kmeans_clustering(vehicles_df)
Use Pricipal Components for Clustering¶
In [540]:
sc= StandardScaler()
df_scaled = sc.fit_transform(vehicles_df.drop("class",axis=1))
do_pca(df_scaled)
P_reduce = np.array(eigvectors_sorted[0:8])
X_reduced = np.dot(df_scaled,P_reduce.T)
X_reduced = pd.DataFrame(X_reduced)
In [541]:
sns.pairplot(X_reduced)
Out[541]:
In [538]:
df=pd.concat([X_reduced,y],axis=1)
df.head()
Out[538]:
In [539]:
df['class'] = df.apply((lambda r: 0 if r["class"] == "van" else 1 if r["class"] == "car" else 2),axis=1)
evaluate_kmeans_clustering(df)
Ensemble Classification and Clustering results¶
NOT IMPLEMENTED EXCEPTION()¶
Summary¶
Non Technical Summary¶
- Cars have higher circularity, primary axis rectangualrity and Compactness
- Vans have low scatter ratio and scaled variance, but high elongateness
- Hollows ratio for "bus" is having lower values compared to"van" and "car"
- max_length_aspect ratio is an important feature for identifying bus from other two classes.
Technical Summary¶
SVM performed very well with this dataset.
The following activities are done to make the model performance better:
1. Tuned hyperparameters (C = 0.1, kernel= rbf)
2. Model is trained and tested over Principal Components(top 8)
3. Model property:
F1 Score :
bus 0.98
car 0.95
van 0.90
Standard Deviation: 1.3
The model predicts the class with 98% precision
In [ ]: